Haptic sensations as a function of eye gaze

ABSTRACT

Systems, electronic devices, and methods provide haptic sensations as a function of eye gaze. A system may include a detector configured to determine a direction of an eye gaze of a user of the system, a processor configured to generate signal representative of a haptic effect based on the direction of the eye gaze, and a haptic output device configured to receive the signal from the processor and output the haptic effect to the user. A method for providing a haptic effect to a user of a system may include determining a direction of an eye gaze of the user of the system, generating a haptic effect based on the direction of the eye gaze, and outputting the haptic effect to the user.

FIELD

The present invention is related to systems, electronic devices, andmethods for providing haptic sensations as a function of eye gaze.

BACKGROUND

Detection methods in order to determine a direction of an eye gaze of auser of an electronic device are becoming more prevalent in humancomputer interactions. Eye gaze detection, for example, may be used tobetter determine where a user is pointing at a display when using theMicrosoft's KINECT® Gaming System or Sony's PLAYSTATION EYE® camera foruse with a computer game console. In general, the user's eye gaze may beused as an input to the computer device. For example, the current focusof the user interface may be directed to where the user is looking,similar to the use of a mouse as a pointer on a computer screen. Assystems like the KINECT® get more sophisticated, eye gaze detection maybecome a standard means of providing input to software applications.

SUMMARY

It is desirable to use eye gaze direction information as an input tocreate haptic effects in an electronic device so that the haptic effectsmay be better matched to the user's overall experience according to whatthe user is currently looking at. This may provide a more immersiveexperience to the user.

According to an aspect of the present invention, there is provided asystem that includes a detector configured to determine a direction ofan eye gaze of a user of the system, a processor configured to generatesignal representative of a haptic effect based on the direction of theeye gaze, and a haptic output device configured to receive the signalfrom the processor and output the haptic effect to the user.

According to an aspect of the present invention, there is provided amethod for providing a haptic effect to a user of a system. The methodincludes determining a direction of an eye gaze of the user of thesystem, generating a haptic effect based on the direction of the eyegaze, and outputting the haptic effect to the user.

According to an aspect of the present invention, there is provided anelectronic device that includes a housing, a detector supported by thehousing, the detector configured to determine a direction of an eye gazeof a user of the electronic device, a haptic output device supported bythe housing, the haptic output device configured to generate a hapticeffect to the user, and a processor supported by the housing. Theprocessor is configured to generate a first haptic drive signalrepresentative of a first haptic effect based on the eye gaze of theuser corresponding to the user looking at the electronic device, and asecond haptic drive signal representative of a second haptic effectbased on the eye gaze of the user corresponding to the user looking awayfrom the electronic device. The processor is configured to output thefirst haptic drive signal or the second haptic drive signal to thehaptic output device to generate the haptic effect based on thedirection of the eye gaze determined by the detector.

BRIEF DESCRIPTION OF THE DRAWINGS

The components of the following Figures are illustrated to emphasize thegeneral principles of the present disclosure and are not necessarilydrawn to scale. Reference characters designating correspondingcomponents are repeated as necessary throughout the Figures for the sakeof consistency and clarity.

FIG. 1 is a schematic illustration of a system for providing hapticsensations to a user of the system as a function of eye gaze of the userin accordance with an embodiment of the invention;

FIG. 1A is a schematic illustration of an embodiment of a detector ofthe system of FIG. 1;

FIG. 1B is a schematic illustration of an embodiment of a detector ofthe system of FIG. 1;

FIG. 2 is a schematic illustration of an embodiment of a processor ofthe system of FIG. 1;

FIG. 3 is a schematic illustration of an embodiment of the system ofFIG. 1;

FIG. 4 is a schematic illustration of an electronic device for providinghaptic sensations to a user of the device as a function of eye gaze ofthe user in accordance with an embodiment of the invention;

FIG. 5 is a schematic illustration of the electronic device of FIG. 5 inaccordance with an embodiment of the invention;

FIG. 6 is a schematic illustration of the electronic device of FIG. 5 inaccordance with an embodiment of the invention;

FIGS. 7A and 7B are schematic illustrations of the electronic device ofFIG. 5 in accordance with an embodiment of the invention;

FIG. 8 is a schematic illustration of the system of FIG. 1 illustratesin accordance with an embodiment of the invention; and

FIG. 9 is a flow diagram of a method for providing haptic sensations toa user of the system of FIG. 1 or the electronic device of FIG. 4 as afunction of eye gaze of the user in accordance with an embodiment of theinvention.

DETAILED DESCRIPTION

FIG. 1 illustrates a system 100 in accordance with an embodiment of theinvention. As illustrated, the system 100 includes a detector 110, aprocessor 120, a haptic output device 130, and a display 140. Thedetector 110 is configured to determine a direction of an eye gaze of auser of the system 100, the processor 120 is configured to generate asignal representative of a haptic effect based on the direction of theeye gaze, the haptic output device 130 is configured to receive thesignal from the processor and output the haptic effect to the user, andthe display 140 is configured to display content to the user.

The detector 110 may include any detection means that are used to detecteye gaze. For example, FIG. 1A illustrates an embodiment of a detector110′ that may include a camera 112 configured to capture an image of aneye of the user of the system 100, and a processor 114 configured todetermine the direction of the eye gaze based on the image. In anembodiment, the processor 114 may be part of the processor 120 ofFIG. 1. Image processing techniques to determine eye gaze direction andare well known in the literature and therefore are not described herein.FIG. 1B illustrates an embodiment of a detector 110″ that may include asensor 116 configured to monitor movements of muscles near the eye ofthe user, and a processor 118 configured to determine the direction ofthe eye gaze based on the monitored movement. In an embodiment, thesensor 116 may be configured to measure electrical activity of themuscles moving the eyes. In an embodiment, the processor 118 may be partof the processor 120 of FIG. 1.

The illustrated embodiments of the detector are not considered to belimiting in any way and other detection means that provide for thedetermination of a direction of the user's eye gaze may be used inaccordance with embodiments of the present invention. For example, in anembodiment, the user's eye gaze direction may be estimated by analyzingthe user's body or head posture.

In an embodiment, the detector 110 may also be configured to determinewhere the user's current eye gaze direction is focused. This may beaccomplished by using image processing techniques to determine theposition and the shape of the iris of a user's eye, in combination witha model or stored reference image of the iris. In an embodiment, theuser's eye gaze direction may be stored as pitch and yaw angles for eacheye. With this information, the depth of field of the user's currentgaze may also be determined. In an embodiment, other sensors may be usedin addition to the detector 110 to better determine the user's intent orvolition, such as sensors that are typically associated with functionalmagnetic resonance imaging (“fMRI”) or electroencephalogram (“EEG”).Haptic effects may be rendered as a function of these combined sensorand detector outputs.

FIG. 2 illustrates an embodiment of the processor 120 in more detail.The processor 120 may be configured to execute one or more computerprogram modules. The one or more computer program modules may includeone or more of a content provision module 122, an eye gaze determinationmodule 124, a haptic effect determination module 126, a haptic outputdevice control module 128, and/or other modules. The processor 120 maybe configured to execute the modules 122, 124, 126, and/or 128 bysoftware, hardware, firmware, some combination of software, hardware,and/or firmware, and/or other mechanisms for configuring processingcapabilities on processor 120.

It should be appreciated that although modules 122, 124, 126, and 128are illustrated in FIG. 3 as being co-located within a single processingunit, in embodiments in which the processor 120 includes multipleprocessing units, one or more of modules 122, 124, 126, and/or 128 maybe located remotely from the other modules. For example, the eye gazedetermination module 124 may reside in the processors 114, 118 describedabove. The description of the functionality provided by the differentmodules 122, 124, 126, and/or 128 described below is for illustrativepurposes, and is not intended to be limiting, as any of the modules 122,124, 126, and/or 128 may provide more or less functionality than isdescribed. For example, one or more of the modules 122, 124, 126, and/or128 may be eliminated, and some or all of its functionality may beprovided by other ones of the modules 122, 124, 126, and/or 128. Asanother example, the processor 120 may be configured to execute one ormore additional modules that may perform some or all of thefunctionality attributed below to one of the modules 122, 124, 126,and/or 128.

The content provision module 122 is configured to control the provisionof content to the user of the system 100 via the display 140. If thecontent includes computer generated images (e.g., in a videogame,virtual world, augmented reality virtual world, simulation, etc.), thecontent provision module 122 is configured to generate the images and/orviews for display to the user through the display 140. If the contentincludes video and/or still images, the content provision module 122 isconfigured to access the video and/or still images and to generate viewsof the video and/or still images for display on the display 140. If thecontent includes audio content, the content provision module 122 isconfigured to generate the electronic signals that will drive a speaker,which may be part of the display 140, to output corresponding sounds.The content, or information from which the content is derived, may beobtained by the content provision module 122 from an electronic storage129, which may be part of the processor 120, as illustrated in FIG. 2,or may be separate from the processor 120.

The eye gaze determination module 124 is configured to determine adirection of the eye gaze of the user based on information from theoutput signals generated by the detector 110. The information related todirection of the user's eye gaze determined by the eye gaze directiondetermination module 124 may describe the direction as a vector in anabsolute coordinate system, with respect to other objects, and/or inother contexts. Such information may include, without limitation,coordinates and/or angular relationships, such as pitch and yaw angles,as described above.

The haptic effect determination module 126 is configured to determinethe haptic effect or sensation to be generated by the haptic outputdevice 130 for the user, based on information received from the detector110 and any other sensor that is configured to determine the user'sintent or volition, as described above. The gaze parameters determinedby the eye gaze determination module 124 may be used alone or inconjunction with other inputs such as content events, motion of anyportion of the system 100, etc. Determining the haptic effect mayinclude determining one or more parameters that include an amplitude,frequency, duration, etc., of the haptic sensation. The haptic effect isdetermined by the haptic effect determination module 126 to enhance oneor more aspects of the experience provided by the content to the user.For example, the haptic effect may be determined to enhance one or moreof the realism of the content, the enjoyability of content, perceptionof the content by the user, and/or other aspects of the experienceprovided by the content being conveyed to the user via the display 140.

The haptic output device control module 128 is configured to control thehaptic output device 130 to generate the haptic effect determined byhaptic effect determination module 126. This includes communicating thehaptic output signal to be generated by the processor 120 andcommunicated the haptic output device 130. The haptic effect to begenerated may be communicated over wired communication links, wirelesscommunication links, and/or other communication links between theprocessor 120 and the haptic output device 130. In an embodiment, atleast a portion of the functionality attributed to the haptic outputdevice control module 128 may be disposed in a processor carried by thehaptic output device 130.

The haptic effects or sensations can be created with any of the methodsof creating haptics, such as vibration, deformation, kinestheticsensations, electrostatic or ultrasonic friction, etc. In an embodiment,the haptic output device 130 may include an actuator, for example, anelectromagnetic actuator such as an Eccentric Rotating Mass (“ERM”) inwhich an eccentric mass is moved by a motor, a Linear Resonant Actuator(“LRA”) in which a mass attached to a spring is driven back and forth,or a “smart material” such as piezoelectric materials, electro-activepolymers or shape memory alloys, a macro-composite fiber actuator, anelectro-static actuator, an electro-tactile actuator, and/or anothertype of actuator that provides a physical feedback such as a haptic(e.g., vibrotactile) feedback. The haptic output device 130 may includenon-mechanical or non-vibratory devices such as those that useelectrostatic friction (ESF), ultrasonic surface friction (USF), orthose that induce acoustic radiation pressure with an ultrasonic haptictransducer, or those that use a haptic substrate and a flexible ordeformable surface, or those that provide projected haptic output suchas a puff of air using an air jet, and so on. In an embodiment, multiplehaptic output devices may be used to generate different haptic effects.The haptic output device 130 may be located in a gamepad, a mobilephone, a wearable device, or any other device that may be used to createthe haptic effects or sensations to the user.

FIG. 3 illustrates a system 300 in accordance with embodiments of theinvention. As illustrated, the system 300 includes a camera based gamingsystem 302, such as a KINECT® gaming system by Microsoft or aPLAYSTATION® gaming system by Sony that is equipped with a PLAYSTATIONEYE® camera. In the illustrated embodiment, the gaming system 302includes the detector 110, which may include the detector 110′ and/orthe detector 110″ described above, and the processor 120 describe above.The system 300 also includes a user input device 304, such as a gamepad,configured to receive an input from a user to provide a command to thegaming system 302. The user input device 304 may also include the hapticoutput device 130 described above. The system 300 also includes thedisplay 140, such as a television or monitor, configured to displayimages output by the content provision module 122 of the gaming system302.

As illustrated in FIG. 3, the user input device 304 is in communicationwith the gaming system 302 though a wired or wireless communicationchannel 350 established between a communication port 306 of the gamingsystem 302 and a communication port 308 of the user input device 304.The haptic output device 130 is configured to receive the signal fromthe processor 120, such as from the haptic output device control module128, and output a haptic effect to the user through the user inputdevice 304. The haptic sensation the user feels may depend on thelocation of the display 140 the user is currently looking at. Forexample, the user may be looking at the current scene on the display140, which is not necessarily where the current gamepad focus islocated.

For example, if a user is playing a first person shooter game and entersa new room, the user uses his/her eye gaze to quickly look around thescene on the display 140. When the user's eye gaze, represented by EG1,falls on a first portion of the room on the left side of the display140, represented by area “A”, a first haptic effect or sensation may befelt in the user input device 304 to indicate the presence of an objectthat can be interacted with. When the user's eye gaze, represented byEG2, falls on a second portion of the room on the right side of thedisplay 140, represented by area “B”, which does not contain an objectthat can be interacted with, the user may feel a second haptic effect orsensation that is different from the first haptic effect of sensation.The second haptic effect may be configured to communicate to the userthat the second portion of the room does not contain an object that canbe interacted with, so that the user does not need to continue to lookat the second portion of the room.

In an embodiment, the user input device 304 may include a gamepad with atouch screen or touch pad input. The user may play a game using thedisplay 140 as a main display and the touch screen on the user inputdevice 304. The detector 110 may detect the eye gaze of the user anddetermine whether the user is looking at the display 140 or whether theuser is looking at the touch screen on the user input device 104. Theprocessor 120 may output a signal to the haptic output device 130 basedon the detected direction of the user's eye gaze so that the hapticeffect generated by the haptic output device 130 may communicate to theuser which screen the user should be looking at. In an embodiment, thehaptic effect may communicate to the user that something has occurred onthe display or screen that the user is not currently looking at todirect the user to look at the other display or screen.

In an embodiment, the user input device 304 may include a wearabledevice, such as a GLASS® augmented reality display by Google Inc. Inthis embodiment, the detected eye gaze may be used to indicate what theuser would like to select on an augmented reality visual display. Forexample, the user may look at a menu item or icon and then verbally say“select” or even blink for selection of the menu item or icon. Thehaptic output device 130 may output haptic effects as the user's gazemoves across menu elements or icons.

In an embodiment, an augmented reality virtual world provided by thecontent provision module 122 may be combined with the eye gaze directioninformation determined by the eye gaze direction determination module124 to determine that a user is looking at a specific object within theaugmented reality virtual world at a certain depth in the visual field.The haptic effect determination module 126 may identify the type ofobject and determine the type of haptic effect to be output by thehaptic output device 130. For example, in an embodiment, a user may playa game in which he/she must find real or virtual objects in an augmentedreality type display. The user's eye gaze direction may be determinedand if the user is looking at object X, a haptic effect associated withobject X, i.e. object X's haptic signature, may be generated on the userinput device 304 by the haptic output device 130.

In an embodiment, the display 140 may be part of a personal computer.The detector 110 may track the user's eye gaze as the user navigates thedisplay 140 with his/her eyes to control a pointer location on thedisplay 140 in a similar manner as a user would navigate a display witha mouse. The haptic output device 130 may output a haptic effect as thedetector 110 detects that the user is looking at each item on thedisplay 140 or when the user is looking at certain highlighted items onthe display 140.

In an embodiment, the system 100 of FIG. 1 may be part of a singleelectronic device. FIG. 4 is a schematic illustration of an electronicdevice 400 in accordance with an embodiment of the invention. Asillustrated, the electronic device 400 includes a processor 410, amemory device 420, and input/output devices 430, which areinterconnected via a bus 440. In an embodiment, the input/output devices430 may include a touch screen device 450 or other human-computerinterface devices.

The touch screen device 450 may be configured as any suitablehuman-computer interface or touch/contact surface assembly. The touchscreen device 450 may be any touch screen, touch pad, touch sensitivestructure, computer monitor, laptop display device, workbook displaydevice, kiosk screen, portable electronic device screen, or othersuitable touch sensitive device. The touch screen device 450 may beconfigured for physical interaction with a user-controlled device, suchas a stylus, finger, etc. In some embodiments, the touch screen device450 may include at least one output device and at least one inputdevice. For example, the touch screen device 450 may include a visualdisplay and a touch sensitive screen superimposed thereon to receiveinputs from a user's finger. The visual display may include a highdefinition display screen.

In various embodiments, the touch screen device 450 is configured toprovide haptic feedback to at least a portion of the electronic device400, which can be conveyed to a user in contact with the electronicdevice 400. Particularly, the touch screen device 450 can provide hapticfeedback to the touch screen itself to impose a haptic effect when theuser in is contact with the screen. The haptic effects can be used toenhance the user experience, and particularly can provide a confirmationto the user that the user has made sufficient contact with the screen tobe detected by the touch screen device 450.

The electronic device 400 may be any device, such as a desktop computer,laptop computer, electronic workbook, electronic handheld device (suchas a mobile phone, gaming device, personal digital assistant (“PDA”),portable e-mail device, portable Internet access device, calculator,etc.), kiosk (such as an automated teller machine, ticking purchasingmachine, etc.), printer, point-of-sale device, game controller, or otherelectronic device.

The processor 410 may be a general-purpose or specific-purpose processoror microcontroller for managing or controlling the operations andfunctions of the electronic device 400. For example, the processor 410may be specifically designed as an application-specific integratedcircuit (“ASIC”) to control output signals to a driver of theinput/output devices 430 to provide haptic effects. In an embodiment,the processor 410 includes the processor 120 described above. Theprocessor 410 may be configured to decide, based on predefined factors,what haptic effects are to be generated, the order in which the hapticeffects are generated, and the magnitude, frequency, duration, and/orother parameters of the haptic effects. The processor 410 may also beconfigured to provide streaming commands that can be used to drive ahaptic output device for providing a particular haptic effect. In someembodiments, the processing device 410 may actually include a pluralityof processors, each configured to perform certain functions within theelectronic device 400.

The memory device 420 may include one or more internally fixed storageunits, removable storage units, and/or remotely accessible storageunits. The various storage units may include any combination of volatilememory and non-volatile memory. The storage units may be configured tostore any combination of information, data, instructions, software code,etc. More particularly, the storage devices may include haptic effectprofiles, instructions for how the haptic output device of theinput/output devices 430 are to be driven, or other information forgenerating haptic effects.

In addition to the touch screen device 450, the input/output devices 430may also include specific input mechanisms and output mechanisms. Forexample, the input mechanisms may include such devices as keyboards,keypads, cursor control devices (e.g., computer mice), or other dataentry devices. The input/output devices 450 may include the detector110, such as the detector 110′ that includes the camera 112 and theprocessor 114 described above. In an embodiment, the user's eye gazethat is detected by the detector 110 may be used to control a pointerlocation on the touch screen device 450, as described above.

Output mechanisms may include a computer monitor, virtual realitydisplay device, audio output device, printer, or other peripheraldevices, such as the gamepad described above with respect the userinterface device 304 of FIG. 3. The input/output devices 430 may includemechanisms that are designed to not only receive input from a user, butalso provide feedback to the user, such as many examples of touch screendevices.

The touch screen device 450 and other input/out devices 430 may includeany suitable combination and configuration of buttons, keypads, cursorcontrol devices, touch screen components, stylus-receptive components,or other data entry components. The touch screen device 450 may alsoinclude any suitable combination of computer monitors, display screens,touch screen displays, haptic output devices, such as the haptic outputdevice 130 described above, or other notification devices for providingoutput to the user.

In an embodiment, the touch screen device 450 includes a display surface460, which may be rigid, that is configured to modulate its frictionproperties through, including but not limited to, electrostatic frictionand ultra-sonic surface vibration, to give the user a feeling of surfacerelief (e.g., hills and valleys) when running a finger or stylus acrossthe display that corresponds to the display image.

FIG. 5 illustrates an embodiment of an electronic device 500, such asthe electronic device 400 described above. As illustrated, theelectronic device 500, which may be a mobile phone, includes a housing502 having a first side 504 and a second side 506, a first camera 510located on the first side 504 of the housing 502 and configured to facethe user during normal operation, and a second camera 512 located on thesecond side 506 of the housing 502 and configured to face away from theuser during normal operation. The first camera 510 may be part of thedetector 110′ described above and may be configured to track thedirection of the user's eye gaze EG. The second camera 512 may be usedto capture an image 514 that includes a real physical object RO. Thedevice 500 also includes a display 540 that is supported by the housing502 and is configured to project an image 516 from the first side 504 ofthe housing 502. The image 516 displayed by the display may be the image514 captured by the second camera 512.

As illustrated in FIG. 5, a processor 520 may be located within andsupported by the housing 502, and may be an embodiment of the processor120 described above. The processor 520 may be configured to augment theimage 514 captured by the second camera 512 with a virtual object VO sothat the image 516 displayed by the display 540 includes representationsof the real object RO and the virtual object VO. A haptic output device530, such as the haptic output device 130 described above, may belocated in and/or supported by the housing 502. In an embodiment, afirst haptic effect may be generated by the haptic output device 530 ifthe processor 520 determines that the eye gaze EG of the user is in thedirection of the real object RO, and a second haptic effect that isdifferent from the first haptic effect may be generated by the hapticoutput device 530 if the processor 520 determines that the eye gaze EGof the user is in the direction of the virtual object VO. In anembodiment, the cameras 510, 512 of the device 500 may be used in asimilar manner to determine the user's eye gaze for navigation purposes.

In an embodiment, the haptic effect that is generated by the hapticoutput device 530 may depend on whether the user's eye gaze is directedtowards or away from the display 540 of the device 500, as detected bythe camera 510 and determined by the processor 520. For example, in anembodiment, an intensity of the haptic effect associated with contactwith a widget or button 560 on the device 500 may be increased by thehaptic output device 530 when the processor 520 determines that the eyegaze of the user is directed away from the screen 540, as indicated byEGA in FIG. 6. This may assist the user of the electronic device 500when trying to operate the widget or button 560 without looking at thedevice 500. When the processor 520 determines that the eye gaze of theuser is directed towards the screen 540, as indicated by EGT in FIG. 6,an intensity of the haptic effect may be decreased, because it isassumed that the user can see the widget or button 560. The amplitudesof the periodic signals in FIG. 6 represent different intensities ofhaptic effects associated with the two different eye gazes EGA, EGT.

In an embodiment, haptic effects may also be used to attract attentionto a notification or other onscreen event, such as the arrival of a newemail. Visual attention of the user with respect to the display 540 ofthe device 500 may be monitored with the camera 510 and the processor520 may determine whether a haptic notification in the form of a hapticsensation generated by the haptic output device 530 is necessary orwhether the user has looked at the display 540 to review thenotification of other onscreen event. For example, as illustrated inFIG. 7A, if it is determined that the user has an eye gaze EGA away fromthe display 540, a haptic effect may be generated by the haptic outputdevice 530 to notify the user that a new email has been received. Thehaptic effect may initially be very strong, as indicated in FIG. 7A, butmay gradually decrease in intensity and fade away once the camera 510and the processor 520 determine that the user is known to have looked ata notification icon 570 or the display 540 by the detection of the eyegaze EGT towards the notification icon 570 or the display 540, asillustrated in FIG. 7B, since the visual feedback is sufficient toprovide the notification.

FIG. 8 illustrates an embodiment in which an electronic device 800 isprovided in the form of a vehicle's onboard computer. The device 800 mayinclude a detector, such as the detector 110′ described above, thatincludes a camera 810 and a processor 820, such as the processor 120described above. In this embodiment, the camera 810 and the processor820 are also configured to monitor the number of glances the user makesto a display 840 of the device 800 during a predetermined amount oftime. The haptic effects provided by a haptic output device 830, such asthe haptic output device 130 discussed above, may be tuned to reduce thenumber of glances the user makes to the display 840. The haptic outputdevice 830 may be located at the display 840, if the display 840includes a touch screen, and/or at another location within the vehiclethat the driver contacts, such as a steering wheel, SW, or a surface ofthe dashboard DB, or the seat on which the driver sits. In anembodiment, if it is determined that the user is glancing at the display840, for example, more than ten times during a minute, an intensity ofthe haptic effect provided by the haptic output device 830 may beincreased, based on the assumption that the user is needing to rely onvisual confirmation because the haptic feedback being provided by thedevice 800 is not strong enough to provide a satisfactory confirmation.Similarly, if it is determined that the user is glancing at the display840, for example, less than two times per minute, the intensity of thehaptic effect may be decreased.

FIG. 9 illustrates a method 900 in accordance with embodiments of theinvention described herein. As illustrated, the method starts at 902. As904, a direction of an eye gaze of a user of a device or a system isdetermined. At 906, a haptic effect is generated based on the directionof the eye gaze. At 908, the haptic effect is output to the user. Themethod ends at 912.

The illustrated and above-described embodiments are not considered to belimiting in any way, and embodiments of the present invention and may beused to enable haptic feedback in various electronic devices, such astouch screen handheld devices (mobile devices, PDA, and navigationsystems), automotive applications, gaming consoles, etc.

Although many of the examples described herein relate to touch screendevices, it should be understood that the present disclosure alsoencompasses other types of human-computer interfaces involving touchsensitive structures. In addition, other features and advantages will beapparent to one of ordinary skill in the art upon reading andunderstanding the general principles of the present disclosure. Theseother features and advantages are intended to be included in the presentdisclosure as well.

The embodiments described herein represent a number of possibleimplementations and examples and are not intended to necessarily limitthe present disclosure to any specific embodiments. Instead, variousmodifications can be made to these embodiments as would be understood byone of ordinary skill in the art. Any such modifications are intended tobe included within the spirit and scope of the present disclosure andprotected by the following claims.

What is claimed is:
 1. A system comprising: a detector configured todetermine a direction of an eye gaze of a user of the system relative toan object on a display device associated with the system; a processorconfigured to determine one or more parameters of a first haptic effectand a second haptic effect, different from the first haptic effect, forthe user based on the direction of the eye gaze relative to the objecton the display device, generate a first haptic output signalrepresentative of the first haptic effect when the eye gaze of the useris determined to be directed at the object on the display device, andgenerate a second haptic output signal representative of the secondhaptic effect when the eye gaze of the user is determined to not bedirected at the object on the display device and directed at anotherobject on the display device that is different from the object, whereinthe object is a real object and the other object is a virtual objectdisplayed on the display device and wherein the one or more parametersselected from the group consisting of an amplitude, a frequency, andduration of the haptic effect; and a haptic output device configured toreceive the first haptic output signal and the second haptic outputsignal from the processor and output the first haptic effect or thesecond haptic effect to the user based on the direction of the eye gazeof the user relative to the object on the display device.
 2. The systemaccording to claim 1, wherein the haptic output device comprises anactuator configured to output the haptic effect.
 3. The system accordingto claim 1, wherein the haptic output device comprises a non-mechanicaldevice configured to output the haptic effect.
 4. The system accordingto claim 1, wherein the detector comprises a camera configured tocapture an image of an eye of the user, and an image processorconfigured to determine the direction of the eye gaze based on theimage.
 5. The system according to claim 1, wherein the detectorcomprises a sensor configured to monitor movements of muscles near aneye of the user, and a second processor configured to determine thedirection of the eye gaze based on the monitored movements.
 6. Thesystem according to claim 1, wherein the detector and the processor arepart of a first device and the haptic output device is part of a seconddevice that is separate from the first device.
 7. The system accordingto claim 6, further comprising a communication port configured toestablish a wired or wireless communication channel between the firstdevice and the second device.
 8. The system according to claim 1,wherein the detector is part of a first device and the processor and thehaptic output device are part of a second device that is separate fromthe first device.
 9. The system according to claim 8, further comprisinga communication port configured to establish a wired or wirelesscommunication channel between the first device and the second device.10. The system according to claim 1, wherein the processor is furtherconfigured to generate an augmented reality space on the display device,and use the direction of the eye gaze to determine when the object theuser is looking at is an object in the augmented reality space.
 11. Amethod for providing a haptic effect to a user of a system, the methodcomprising: determining a direction of an eye gaze of the user of thesystem relative to an object on a display device with a detector;determining one or more parameters of a first haptic effect and a secondhaptic effect, different from the first haptic effect, for the userbased on the direction of the eye gaze relative to the object on thedisplay device with a processor, the one or more parameters selectedfrom the group consisting of an amplitude, a frequency, and duration ofthe haptic effect; generating a first haptic output signalrepresentative of the first haptic effect when the eye gaze of the useris determined to be directed at the object on the display device,generating a second haptic output signal representative of the secondhaptic effect when the eye gaze of the user is determined to not bedirected at the object on the display device and directed at anotherobject on the display device that is different from the object, whereinthe object is a real object and the other object is a virtual objectdisplayed on the display device; and outputting the first haptic effectto the user with a haptic output device based on the first haptic outputsignal or the second haptic effect to the user with the haptic outputdevice based on the second haptic output signal based on the directionof the eye gaze of the user relative to the object on the displaydevice.
 12. The method according to claim 11, wherein said determiningcomprising capturing an image of an eye of the user with the detector,and determining the direction of the eye gaze based on the image withthe processor.
 13. The method according to claim 11, wherein saiddetermining comprises monitoring movements of muscles near an eye of theuser with the detector, and determining the direction of the eye gazebased on the monitored movements with the processor.
 14. The methodaccording to claim 11, wherein said second haptic effect is generatedwhen the eye gaze of the user is determined to be directed at anotherobject on the display device that is different from the object.
 15. Anelectronic device comprising: a housing; a detector supported by thehousing, the detector configured to determine a direction of an eye gazeof a user of the electronic device relative to an object on a displaydevice associated with the electronic device; a haptic output devicesupported by the housing, the haptic output device configured togenerate a haptic effect to the user; and a processor supported by thehousing, the processor configured to generate: i) a first haptic drivesignal representative of a first haptic effect based on the eye gaze ofthe user corresponding to the user looking at the object on the displaydevice associated with the electronic device, and ii) a second hapticdrive signal representative of a second haptic effect based on the eyegaze of the user corresponding to the user looking away from the objecton the display device associated with the electronic device and atanother object on the display device that is different from the object,wherein the object is a real object and the other object is a virtualobject displayed on the display device, the second haptic effect havingone or more parameters different from the first haptic effect, the oneor more parameters selected from the group consisting of an amplitude, afrequency, and duration of the haptic effect, the processor furtherbeing configured to output the first haptic drive signal or the secondhaptic drive signal to the haptic output device to generate the hapticeffect based on the direction of the eye gaze relative to the object onthe display device associated with the electronic device determined bythe detector.
 16. The electronic device according to claim 15, whereinthe detector comprises a camera configured to capture an image of an eyeof the user, and an image processor configured to determine thedirection of the eye gaze based on the image.
 17. The electronic deviceaccording to claim 15, wherein the display device is supported by thehousing and the display device is configured to display images.
 18. Theelectronic device according to claim 17, wherein the display deviceincludes a touch screen configured to receive input from the user. 19.The electronic device according to claim 15, wherein an intensity of thesecond haptic effect is greater than an intensity of the first hapticeffect.
 20. The electronic device according to claim 15, wherein theelectronic device is a mobile phone.
 21. The system according to claim1, wherein the system is a gaming system.
 22. The system according toclaim 1, wherein the system is part of a vehicle.
 23. The methodaccording to claim 11, further comprising: generating an augmentedreality space that includes the object; displaying the augmented realityspace on the display device; and generating the first haptic outputsignal when the eye gaze of the user is determined to be directed at theobject in the augmented reality space.
 24. The system according to claim1, wherein the direction of the eye gaze of the user is usable forproviding a command to the processor and the processor is furtherconfigured to: determine one or more parameters of a third haptic effectbased on the command; and generate a third haptic output signalrepresentative of the third haptic effect, wherein the haptic outputdevice is further configured to receive the third haptic output signaland output the third haptic effect to the user.
 25. The method accordingto claim 11, further comprising: determining a user command based on thedirection of the eye gaze of the user; determining one or moreparameters of a third haptic effect based on the user command;generating a third haptic output signal representative of the thirdhaptic effect; and outputting the third haptic effect to the user withthe haptic output device based at least in part on the third hapticoutput signal and the user command.